Dental shade mapping

ABSTRACT

A method and apparatus for obtaining a color mapping of a dental object. Illumination is directed toward the object over at least first, second, and third wavelength band, one band at a time. An image of the dental object is captured at each wavelength band to form a set of images of the dental object. For pixels in the captured set of images, an image data value for the pixel corresponds to each of the wavelength bands and calculates interpolated image data values proportional to the spectral reflectance of the dental object, according to the obtained image data values and according to image data values obtained from a reference object at the wavelength bands. Spectral distribution data for a viewing illuminant is obtained and the visual color of the dental object reconstructed according to the calculated interpolated image data values and the obtained spectral distribution of the viewing illuminant.

CROSS REFERENCE TO RELATED APPLICATIONS

Reference is made to U.S. Ser. No. 12/834,921, entitled “DENTAL SHADEMAPPING”, filed on 13 Jul. 2010 in the names of Wong et al., whichpublished as US 2012/0014571, and which is commonly assigned.

FIELD OF THE INVENTION

This invention relates generally to methods and systems for dental colormeasurement and more particularly relates to a digital method and systemfor determining color shade information for natural teeth, referenceshade samples, and fabricated dental prostheses.

BACKGROUND OF THE INVENTION

Modern restorative dental procedures often require accurate colormatching, such as for filling materials and for the fabrication ofrestorations such as crowns, implants, fixed partial dentures, andveneers. The materials used for these procedures, such as ceramics andother materials, can be skillfully formed and treated to closely matchthe shape, texture, color and translucency of natural teeth.

A widely used technique for determining and communicating tooth colorinformation is a process referred to as “shade matching” whereby thedentist or technician visually matches a patient's tooth to one of anumber of reference shade samples or shade tabs within one or more setsof standardized shade guides. The practitioner who performs the matchrecords the identification of the matching shade tab and conveys thatinformation to the dental laboratory where the restoration or prosthesisis then fabricated. The laboratory then uses its own set of the sameshade guides to perform visual color evaluations of the restoration orprosthesis throughout the fabrication process.

The visual shade matching process can be highly subjective and subjectto a number of problems. The initial matching procedure is oftendifficult and tedious, and it is not unusual for the process to taketwenty minutes or longer. In many cases, there is no shade tab thatperfectly matches the patient's teeth.

The problem of accurately modeling the color of a tooth is more complexthan obtaining a close color match using shade tabs. The inherentshortcomings and limitations of both instrument-based and visual-basedshade-matching systems can be more fully appreciated by considering thedifficulties involved in matching the appearance of human teeth. Toothcolor itself results from a relatively complex interaction ofreflection, transmission, refraction, fluorescence, and scattering by avariety of organic and inorganic components. It is influenced byvariations in tooth pulp volume, dentin condition, enamel composition,and other variations in the composition, structure, and thickness of thedental tissues. One result of this complexity is that color appearanceand color measurement are greatly influenced by lighting geometry,surrounding colors, and other environmental factors.

As a further complication, color within a single tooth is generally notuniform. Color non-uniformities can result from spatial variations incomposition, structure, thickness, internal and external stains, surfacetexture, fissures, cracks, and degree of wetness. As a result,measurements taken over relatively large areas produce averaged valuesthat may not be representative of a tooth's dominant color. In addition,natural color variations and non-uniformities make it unlikely that agiven tooth can be matched exactly by any single shade tab. This meansthat a method for conveying the distribution of color within a tooth,not just its average color, is required. Further, tooth color is seldomuniform from tooth to tooth. Therefore, the ideal color of a restorationmay not be in visual harmony with that of an adjacent tooth or of anyother single tooth in a patient's mouth. Moreover, people generally areparticular about the appearance of their teeth. Understandably, they arequite intolerant of restorations that appear inappropriate in color.

In cosmetic dentistry, the fabrication lab often requires additionalinformation in order to more accurately map tooth color in addition tosimple shade matching. In practice, the dentist or technician mayprovide a photograph in addition to a shade tab, so that the fabricationlab can adjust color characteristics over different portions of thetooth. This helps to provide a type of color mapping for subjective use,with information that relates to the shade tab and shows how colors inother portions of the tooth vary from that of the shade tab.

It is often difficult to decide which tab matches most closely (or,conversely, which has the least mismatch) and to provide accurateinformation on color variation over the tooth surface. Frequently, thepractitioner determines that the patient's teeth are particularlydifficult to match, requiring that the patient then go in person to theorthodontics laboratory that will be fabricating the restoration. There,trained laboratory personnel can perform the color match and colormapping. In many cases, the patient may even need to return to thedentist and laboratory two, three, or even more times as the color ofthe prosthesis is fine tuned by sequential additions of ceramics orother colored materials. In a high percentage of cases, estimated to benearly 10% for some dental prostheses, the visual color matchingprocedure still fails and the prosthesis that has been fabricated isrejected for color or visual harmony by the dentist or by the patient.

Considering the relative difficulty of the color matching task, and thefurther complexity of color mapping, a high rate of failure is not atall surprising. Visual color evaluation of relatively small colordifferences is always difficult, and the conditions under which dentalcolor evaluations must be made are likely to give rise to a number ofcomplicating psychophysical effects such as local chromatic adaptation,local brightness adaptation, and lateral-brightness adaptation.Moreover, shade tabs provide at best a metameric (that is, non-spectral)match to real teeth; thus, the matching is illuminant-sensitive andsubject to variability due to normal variations in human color vision,such as observer metamerism, for example.

In response to the need for improved color matching and color mapping indental applications, a number of approaches have been attempted.Conventional solutions to this problem are of the following generaltypes:

-   -   (i) RGB-based devices. With this approach, an image of the        entire tooth is captured under white light illumination using a        color sensor. Tristimulus values are calculated over areas of        the tooth surface from RGB (Red, Green, Blue) values of the        3-color channels of sensor, making use of a color calibration        transform. Color analysis by RGB-based devices relies heavily on        the quality of the captured image and requires robust        calibration and may require use of the same camera for        color-matching of tooth and prosthetic device. This requirement        can be due to calibration of the camera itself as well as to        color preprocessing that is performed within the camera in order        to provide the RGB data; this preprocessing can vary        significantly from one camera to the next, even for cameras from        the same manufacturer. Maintaining accuracy tends to be        difficult and measurements are compromised due to metamerism, in        which the color measured is highly dependent upon the        illuminant. This is particularly troublesome since dental        measurement and imaging are generally carried out under        conditions that differ significantly from natural lighting        conditions. Examples using RGB measurement and employing a        corresponding color transform in this way include: U.S. Pat. No.        5,766,006 entitled “Tooth Shade Analyzer System and Methods” to        Murljacic; U.S. Pat. No. 6,008,905 entitled “Method and        Apparatus for Determining the Appearance of an Object” to        Breton, et al.; and U.S. Pat. No. 7,064,830 entitled “Dental        Color Imaging System” to Giorgianni et al.    -   (ii) Colorimetric devices. Devices of this type are engineered        to directly measure color as perceived by the human eye. With        this type of device, illuminating light or reflected light        (under white light illumination) is filtered at the three        wavelength bands that correspond to the spectral response        characteristic or color matching functions of the eye, and        measured reflected signals are directly translated into        tristimulus values. As with RGB-based devices described in (i),        measurements from this type of device also suffer from        metamerism. Some examples using this approach include those        disclosed in U.S. Pat. No. 5,383,020 entitled “Method and        Apparatus for Determining the Color of a Translucent Object Such        as a Tooth” to Vieillefosse that requires a spectrometer and        U.S. Pat. No. 6,867,864 entitled “Optical Measurement Device and        Related Process” to Overbeck et al.    -   (iii) Spectrophotometric devices. These devices employ spectral        reflectance for obtaining color data. Illuminating or reflected        light is spectrally scanned, and light reflected by the tooth is        recorded, using a photosensor, as a function of wavelength.        Visual color, that is, CIE (Commission Internationale de        L'Éclairage or International Commission on Illumination)        tristimulus color information, is then calculated from the        measured spectral reflectance curve. Spectrophotometric devices        are not subject to the same tendency to metamerism inherent to        colorimetric and RGB-based devices and, potentially, yield more        accurate color measurements. It is significant to note, however,        that the spectrophotometer is not an imaging device. The        spectrophotometer is an instrument that measures the spectral        content of incoming light over a small area using a photosensor.        Examples of tooth color measurement using spectrophotometric        devices include U.S. Pat. No. 4,836,674 entitled “Method and        Apparatus for Determining Color, in Particular of a Dental        Prosthesis” and U.S. Pat. No. 6,038,024 entitled “Method and        Apparatus for Determining the Color Stimulus Specification of an        Object” to Berner.

Although the data obtained using the spectrophotometric approachprovides advantages for color matching over colorimetric and RGBapproaches, including elimination of metamerism, this approach has beenfound difficult to implement in practice. The use of a light scanningcomponent for measuring different spectral components, generallyemploying a grating or filter wheel, tends to make thespectrophotometric system fairly bulky and complex. This makes itdifficult to measure teeth toward the back of the mouth, for example.Attempts to alleviate this problem have not shown great success. As oneexample, U.S. Pat. No. 5,745,229 entitled “Apparatus for DeterminingOptical Characteristics of an Object” to Jung et al. provides a compactspectrophotometric device employing optical fibers to channel reflectedlight to an array of sensors, each sensor using a different spectralfilter. However, as is true of spectrophotometric devices in general(iii, above), this device measures only a small area of the toothsurface at a time. To obtain a color mapping of an entire tooth surfacerequires numerous separate measurements with this approach. The imagecapture process is time-consuming and does not provide consistentresults. Color mappings can be inaccurate using such an approach, sincethere can be considerable sensitivity to illumination and image captureangles and probe orientation during the imaging process.

In general, conventional methods that employ color filters, either atthe illuminant end or at the sensor end, can be less desirable becausethey are subject to the limitations of the filter itself.

Thus, there is a need for an improved measurement apparatus thatprovides dental shade matching and mapping in a procedure that isstraightforward to execute, having a high degree of accuracy, butwithout high cost or complex components.

SUMMARY OF THE INVENTION

An object of the present invention is to advance the art of color shademapping in dental applications. With this object in mind, the presentinvention provides an apparatus and method that obtains spectralreflectance data from multi-color images of the tooth without thecomplexity of using a spectrophotometer.

An advantage of the present invention is that it employs an imagingarray for obtaining spectrophotometric measurements over the full image.This promotes adaption for intraoral camera use. In addition, the outputspectrophotometric color data that is provided is not subject tometamerism, which affects solutions that use colorimetric and RGB colormatching techniques. The approach employed in particular embodiments ofthe present invention obtains spectral reflectance data for each pixelof the tooth image, allowing accurate mapping of tooth color, includingvariation in color over different portions of the tooth.

These objects are given only by way of illustrative example, and suchobjects may be exemplary of one or more embodiments of the invention.Other desirable objectives and advantages inherently achieved by thedisclosed invention may occur or become apparent to those skilled in theart. The invention is defined by the appended claims.

According to one aspect of the invention, there is provided a method forobtaining a color mapping for a dental object, the method comprising:directing illumination toward the dental object over at least a first, asecond, and a third wavelength band, one wavelength band at a time;capturing, on an imaging array, an image of the dental object at eachillumination wavelength band to form a set of images of the dentalobject; for each of a plurality of pixels in the captured set of images:(i) obtaining an image data value for the pixel corresponding to each ofthe at least first, second, and third wavelength bands; (ii) calculatinga plurality of interpolated image data values proportional to thespectral reflectance of the dental object, according to the obtainedimage data values and according to image data values obtained from areference object at the at least first, second, and third wavelengthbands; obtaining spectral distribution data for a viewing illuminant;reconstructing the visual color of the dental object according to thecalculated interpolated image data values and the obtained spectraldistribution of the viewing illuminant; and storing at least thereconstructed visual color as data in a computer-accessible memory.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features, and advantages of theinvention will be apparent from the following more particulardescription of the embodiments of the invention, as illustrated in theaccompanying drawings. The elements of the drawings are not necessarilyto scale relative to each other.

FIG. 1 is a schematic diagram showing the arrangement of components in aconventional image-based apparatus for obtaining a color measurement fora tooth.

FIG. 2 is a schematic diagram showing an earlier method that makes useof a colorimetric transform for obtaining tooth color values.

FIG. 3 is a schematic block diagram that shows a color mapping apparatusaccording to one embodiment of the present invention.

FIG. 4A is a graph showing intensity measurements related to specificwavelengths.

FIG. 4B is a graph that shows how interpolation is used to generate acurve for points between the measured data of FIG. 4A.

FIG. 4C is a graph showing typical spectral characteristics for lightsources used in one embodiment.

FIG. 4D is a graph showing a spectral response characteristic for abroadband sensor array in one embodiment.

FIG. 5 is a logic flow diagram of steps for obtaining spectral data froma tooth.

FIG. 6 is a graph showing discrete interpolated values related tomeasured values.

FIG. 7A is a schematic diagram showing the tristimulus data obtained foreach pixel using conventional color matching.

FIG. 7B is a schematic diagram showing the spectral reflectance dataobtained for each pixel using the apparatus and methods of the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

The following is a detailed description of the preferred embodiments ofthe invention, reference being made to the drawings in which the samereference numerals identify the same elements of structure in each ofthe several figures.

In the context of the present application, the term “narrow band” isused to describe LED or other illumination sources that emit most oftheir output light over a narrow range of wavelengths, such as 20-50 nmwide. The term “broadband” is used to describe a light sensor thatexhibits high sensitivity to incident light over a wide wavelength rangeextending at least from about 400 nm to about 700 nm. Because this typeof sensor responds to light but does not distinguish color, it is oftenreferred to as a “monochrome” sensor or, somewhat inaccurately, as a“black-and-white” sensor.

In the context of the present application, the term “pixel”, for “pixelelement”, has its common meaning as the term is understood to thoseskilled in the image processing arts. An electronic image of an objectis captured by an array of light-sensitive elements, each of whichprovides the signal for forming a pixel of image data.

Figures shown and described herein are provided in order to illustratekey principles of operation and component relationships along theirrespective optical paths according to the present invention and are notdrawn with intent to show actual size or scale. Some exaggeration may benecessary in order to emphasize basic structural relationships orprinciples of operation. Some conventional components that would beneeded for implementation of the described embodiments, such as varioustypes of optical mounts, for example, are not shown in the drawings inorder to simplify description of the invention itself. In the drawingsand text that follow, like components are designated with like referencenumerals, and similar descriptions concerning components and arrangementor interaction of components already described are omitted. Where theyare used, the terms “first”, “second”, and so on, do not necessarilydenote any ordinal or priority relation, but are simply used to moreclearly distinguish one element from another.

The terms “color” and “wavelength band” are generally synonymous as usedin the context of the present disclosure. For example, a laser or othersolid-state light source is referred to by its general color, such asRed, rather than by its peak output wavelength (such as 635 nm) or itswavelength band (such as 630-640 nm).

The term “set”, as used herein, refers to a non-empty set, as theconcept of a collection of elements or members of a set is widelyunderstood in elementary mathematics. The term “subset”, unlessotherwise explicitly stated, is used herein to refer to a non-emptyproper subset, that is, to a subset of the larger set, having one ormore members. For a set S, a subset may comprise the complete set S. A“proper subset” of set S, however, is strictly contained in set S andexcludes at least one member of set S.

In the context of the present disclosure, the term “dental object”refers to an object, material, or other element for intra-oral use orapplication and includes teeth, prosthetic devices such as crowns,dentures, braces and other supports and bridges, filling materials,shade-matching tabs, and the like.

In contrast to some attempts at characterizing tooth color, theapparatus and methods of the present invention take into account acombination of factors that affect color measurement and that complicatethe task of accurately characterizing color. For example, particularembodiments of the present invention identify and compensate forvariable factors such as illumination wavelengths and detector responsecharacteristics in order to derive accurate color data. To do this, theapproach that is used in embodiments of the present invention obtains,for each pixel in the image of a tooth or other dental object, spectralreflectance data that is substantially independent of the spectralresponse of the measurement device and that can be used to provide anobjective measure of color that applies for illuminant over anycombination of wavelengths. As a result, the data that is obtained fortooth color mapping in the present invention can be used to reconstructthe visual color of an object when viewed under any illuminant takenfrom a set of available illuminants with known spectral distributions.The color mapping that is generated for a dental object can then be usedfor generating a displayed image or used for comparison against colormapping data for another object or material, such as a crown or otherdental prosthetic device or a filling material. The color mapping thatis generated for a dental object can also be used for designing andforming a dental prosthetic device, for example. A color mapping for atooth or other dental object can consist of a considerable amount ofdata and is typically stored as a data file in a computer-accessiblememory.

Referring to FIG. 1, there is shown a schematic block diagram of aconventional imaging apparatus 10 for obtaining dental color data. Anillumination source 12 directs light onto a tooth 20. A capture module18 then performs image capture and provides image data to a colorreconstruction module 22. The output is a set of visual color valuesthat correspond to points on the tooth surface.

The basic arrangement of FIG. 1 is used for each of the image-basedcolor measurement approaches described earlier in the backgroundsection. FIG. 2, for example, shows a schematic diagram for implementingthe system described in the commonly assigned Giorgianni et al. '830patent cited earlier. In an imaging apparatus 30, illumination source 12provides white light illumination to tooth 20. A lens 32 directsreflected light to a sensor 34 that provides corresponding Red, Green,and Blue values for each pixel. A color calibration transform 36 thengenerates visual color values as output for every image point.

In contrast to the approaches shown in FIGS. 1 and 2, the apparatus andmethods of the present invention provide a color mapping apparatus thatutilizes color reconstruction based on spectral reflectance. Thisapproach is advantaged over RGB-based and colorimetric devices byproviding an inherently more accurate set of data on the actual colorcharacteristics of the tooth. Advantageously, this method is not subjectto metamerism, which would otherwise render measurements dependent onthe illuminant of the measuring system. Unlike spectrophotometricmeasurement devices in general, the apparatus and method of the presentinvention obtain spectral data for each pixel in the tooth image.Moreover, this information is obtained using an imaging array ratherthan a photosensor.

The apparatus and methods of the present invention obtain not only anaccurate color measurement for a single pixel or grouping of adjacentpixels, but, because they use an imaging device, also provide suitableinformation for accurate color mapping over the full image of the dentalobject. By obtaining spectral measurement data using an imaging array,embodiments of the present invention obtain measurements that allowspectral data to be generated for each pixel of the tooth image.

Referring to the schematic diagram of FIG. 3, a color measuring andspectral reflectance mapping apparatus 40 uses an illumination apparatus24 that can provide light of separate colors. In one embodiment,illumination apparatus 24 consists of multiple narrow band light sources14 b, 14 g, 14 y, and 14 r, shown as color LEDs, having wavelength bandsλ1, λ2, λ3, and λ4, respectively. In the embodiment of FIG. 3, four LEDsare shown by way of example; there can be any number of different colorsand more than a single LED or other light source for each color. Abroadband imaging sensor array 44, a CCD (charge-coupled device) or CMOS(Complementary Metal-Oxide Semiconductor) imaging array in thisembodiment, provides a set of output values for each pixel correspondingto reflection from each narrow band light source. A reference target 28is optionally provided as a reference object for obtaining referenceintensity measurements used in correcting for intensity fluctuations inthe system, as described subsequently. In one embodiment, referencetarget 28 is a gray patch, with known spectral reflectancecharacteristics. Reference target 28 and tooth 20 are both within thefield of view of imaging lens 32. In an alternate embodiment, referencetarget 28 does not appear in the image field, but is a separate deviceor object used for obtaining reference and calibration data.

In order to obtain the spectral data using the apparatus shown in FIG.3, the LEDs or other light sources can be energized according to colorgroups, one color group at a time in rapid succession, and thecorresponding measurements of reflected light are obtained by imagingsensor array 44. As shown in thumbnail form in FIG. 3, this effectivelyprovides, for each pixel of the tooth image, a number of points on agraph, one intensity reading corresponding to each of wavelength bandsλ1, λ2, λ3, and λ4 in this example.

A spectral reflectance interpolation model 46 then generates the fullspectral reflectance curve using these points and interpolates thepoints between these measured values using one of a number ofcalculation techniques. The result is a spectral reflectance curve foreach individual pixel of the tooth image. This data provides a colormapping that more accurately represents a color shade than do earliermethods that simply attempt to measure tristimulus values directly orperform color conversion from RGB to a standard color space, such ashue-saturation-brightness value (HSV) or Commission Internationale deL'Éclairage L*a*b* (CIELAB) color space, for example. Also shown in FIG.3 is control logic processor 38, which performs control logic processingand contains supporting electronic memory components that execute theseprocessing functions and store interim and final results. Suchcomponents are familiar to those skilled in the imaging arts.

The graph of FIG. 4A shows an example in which four discrete intensitymeasurements are obtained for a pixel, one for each light source 14 b,14 g, 14 y, and 14 r, corresponding respectively to wavelength bands λ1,λ2, λ3, λ4. This yields four points in a graph of intensity vs.wavelength, as shown. The full spectral reflectance characteristic has avalue at each wavelength, however. Thus, some type of interpolation isneeded in order to obtain the complete spectral reflectance curve forthe pixel, as shown by way of example in FIG. 4B.

Taking accurate measurements requires knowledge of different energylevels and measurement sensitivities in the system. As FIG. 4C shows,each narrow-band light source, an LED in this embodiment, provides anoutput intensity over a constrained range. Wavelength bands λ1, λ2, λ3,and λ4 are identified by the nominal wavelengths at which theseintensity curves peak.

Broadband sensor array 44 can be any type of sensor array that measuresthe amount of reflected light that corresponds to the illuminationapparatus 24 component that is energized and that provides a measuredvalue for each pixel. In one embodiment, broadband sensor array 44 has abroad spectral response characteristic over the visible wavelengthrange. This is distinctly different from the “color matching functions”of the eye or the spectral response of a color sensor array, both ofwhich have significant values only in isolated bands of the visiblewavelengths. For example, the monochrome CCD imaging array may have atypical quantum efficiency curve as shown in the example of FIG. 4D.This device measures the amount of reflected light that corresponds toeach light source 14 b, 14 g, 14 y, and 14 r, as it is illuminated andprovides an output signal that is indicative of the sensed lightintensity received. Because the peak wavelength and bandwidth of the LEDlight source is known, the measured value can be readily associated witha wavelength, without the need for a color filter array (CFA) or otherfiltering component.

The logic flow diagram of FIG. 5 shows steps executed by control logicprocessing components in color measuring and spectral reflectancemapping apparatus 40 of FIG. 3 for obtaining spectral reflectance valuesfor a tooth in one embodiment of the present invention. A loopingprocedure executes once per source color, here for each LED color orother narrow-band illumination source. In an illumination step S110, theLED or other narrow-band illumination source is energized. An imagecapture step S120 obtains an image of tooth 20 and, optionally, ofreference target 28 at the given illumination. The resulting imageconsists of light reflected from the tooth, captured for each pixel inimage capture step S120. The reflectance point values are pixel valuesproportional to the actual reflectance values. The array of reflectancepoint values measured from the broadband sensor array is then obtainedand stored in an obtain values step S130. The loop consisting of stepsS110, S120, and S130 repeats until each color group of LEDs or othernarrow-band light source has been energized. The net result is a set ofimages for the tooth or other dental object.

Continuing with the logic flow in FIG. 5, an interpolation step S140then constructs, for each image pixel, additional values that areproportional to spectral reflectance and will be used for generating aspectral reflectance curve, according to stored point values from thepreceding steps. With respect to the schematic block diagram of FIG. 3,this step is executed as part of spectral reflectance interpolationmodel 46. In one embodiment, this data is used to construct the spectralreflectance curve of FIG. 6 for each pixel of the tooth image. Acalculation step S150 then collects and reconstructs visual color valuesfor the tooth region using the interpolated values from step S140,spectral distribution data for a viewing illuminant, and the CIE viewermatching function. The output visual color values can then be used tomore accurately communicate the true color content of the tooth, withoutmetamerism or other effects of the illumination system. These values canbe used for image rendering and can be stored in a computer-accessibleelectronic memory for future use. The memory itself can be arandom-access device, for short-term storage, or an optical or magneticstorage unit, for longer-term storage. The color mapping can begenerated as needed, so that memory storage is a temporary “workspace”,and is used by control logic processor 38 only for the duration ofperforming a color match or comparison, or during display of an imagefor example. Alternately, control logic processor 38 can store a colormapping for a longer term, such as part of a database of color mappings,indexed by illuminant types, for example. Once the color mapping isformed and is available in computer-accessible memory, it can be usedfor comparison against corresponding color mapping data for a prostheticdevice or material, for example.

It is noted that the sequence of FIG. 5 allows color matching under anyof a set of illuminant conditions. The color mapping data that iscollected can be used to calculate the color that is best matched withany source of viewing light, including incandescent light, fluorescentlight, natural light or sunlight, or other source. Thus, for example,the color mapping data that is obtained in a dentist's office can beused to determine the best matched color for prosthesis viewed innatural light (sunlight), or for prosthesis viewed under stage lighting,such as for a model or performer, or for prosthesis viewed influorescent lighting in office environment, or other lightingconditions. This is an advantage of the method of the present inventionover earlier RGB and colorimetric methods in which the color mapping andcolor match were constrained to the specific lighting conditions used inmaking the color measurement, which could be very different from thelighting condition that matters most to the patient.

The information obtained from the process of FIG. 5 can be stored andused in a number of ways. The visual color values that are generated canbe directly used to render the mapping for display, for example, or canbe encoded in some way and stored in memory. Spectral reflectance valuescan be correlated with other tooth image data and provided to a dentallaboratory or other facility for which accurate color characterizationis useful. Mapping interpolation coefficients, described subsequently inmore detail, can alternately be stored along with measured or calculatedpixel values.

The graph of FIG. 6 shows the results of processing in steps S140 andS150 of the logic sequence described with respect to FIG. 5, as appliedto each pixel of the tooth image. Measured values are used to providedata that is proportional to spectral reflectance and that is used todefine the characteristic spectral reflectance curve. A number ofinterpolated values are then provided for wavelengths lying between andoutside the measured values. Using conventional terminology for appliedinterpolation techniques, the measured values shown in FIG. 6 areconsidered to be “knots”, with interpolated values along the segmentsbetween the knots.

Various suitable interpolation methods can be used for determining theinterpolated values. In one embodiment, simple linear interpolation isused, based on a linear relationship of measured values. Other, morecomplex interpolation schemes well known in the art include higher-orderspline-fitting algorithms, such as cubic splines, for example. The morecomplex spline-fitting techniques may use polynomial methods to achievea smoother interpolation curve.

Another interpolation method uses least-squares interpolation. Thismethod seeks to minimize the distance between the values as calculatedby the interpolation algorithm and actual values measured.

It is noted that the method described in FIG. 5 obtains the multiplemeasurements shown in the example of FIG. 6 for each pixel of the toothimage in one embodiment. This is potentially a sizable amount of data,but provides a full characterization of the spectral content of thetooth image, pixel by pixel. The data collected for the tooth image isnon-metameric, thus eliminating the undesirable effects of illuminationdependence that can compromise the color data when conventionalcolorimetric, RGB, or visually matched systems are employed for colormapping.

Calculation Procedure

As noted earlier, the apparatus and method of the present inventiongenerate and use a significant amount of spectral data based on a fewmeasurements. Calculation step S150 of FIG. 5 can be used to generateany type of useful color data information that can be extracted from thearray of spectral reflectance values, and thus to convert this largevolume of data to a more readily usable form. CIE tristimulus X, Y, Zvalues for one or more pixels provide one useful mechanism forrepresenting color data obtained from these measurements according tovisual response characteristics.

As has been noted earlier, the task of obtaining accurate colormeasurement using conventional methods is confounded by variables ofillumination and sensor response. These problems are addressed by theimproved method of the present invention. The generalized measuredsignal S_(i(tooth)) that is obtained for a particular wavelength λ, forLED illumination in the equations that follow, using color measuring andspectral reflectance mapping apparatus 40, can be represented asfollows:S _(i(tooth)) =∫I _(LEDi)(λ)R ₀(λ)D(λ)dλ  (1)wherein R₀ is the actual reflectance of the tooth or other dentalobject. This variable is also expressed, with reference to the toothmaterial, as R_(tooth); D(λ) is the sensor response; I_(LEDi) is theintensity of the i^(th) LED or other narrow-band light source.

It can be observed that equation (1) holds true for color measurement ofthe tooth in general, whether using RGB or white light sources andwhether the image sensor array is a CCD, CMOS, or other type of lightsensing device. It is particularly instructive to note that conventionaltooth color measurement devices, such as RGB-based color measurementdevices and colorimetric measurement devices, generate the tooth colorvalues directly from measured signal S_(i(tooth)). As equation (1)shows, however, this signal is itself the product of three variablefactors: the actual tooth reflectance R₀, the illuminant, and sensorresponse. It is due to the dependency of this measurement on the lightsource and on sensor response that accurate color computation is notobtained with a conventional RGB-based or colorimetric measurementapparatus. However, unlike conventional solutions, the approach of thepresent invention isolates tooth reflectance R₀ from the other twofactors in the measured signal. The obtained reflectance R₀ can then beused to accurately characterize color under any lighting conditions. Inthis way, the method of the present invention employs aspectrophotometric approach to color characterization. However, unlikeconventional spectrophotometric instruments that obtain one or moremeasurements to extract this data for a single photosensor at onelocation on the object, the apparatus and methods of the presentinvention obtain this data using well-characterized narrow-band lightsources with a monochrome sensor array that captures a mapping ofspectrophotometric data for the full object.

In embodiments of the present invention, multiple signal valuesS_(i(tooth)) are obtained, one for each of the N LED color light sourcesindexed i (i=1, 2, . . . , N). As equation (1) shows, in order toprovide accurate and consistent measurement using S_(i(tooth)), it isnecessary to compensate for short-term changes in the device, such asfrom fluctuations in illuminant intensity and/or sensor response D(λ).For this reason, as part of the imaging process, a reference targetmeasurement S_(i(ref)) is optionally obtained for use as calibrationreference:S _(i(ref)) =∫I _(LEDi)(λ)R _(ref)(λ)D(λ)dλ  (2)wherein R_(ref) is the reflectance of reference target 28. This value isobtained using image readings from reference target 28 (FIG. 3), whichis positioned within the same captured field of view as the tooth in oneembodiment. In an alternate embodiment, image readings from referencetarget 28 are obtained separately, such as at the beginning of animaging session, for example.

Given values S_(i(tooth)) and S_(i(ref)), the more useful quantity forobtaining spectral reflectance is the corrected measurement value isgiven by:

$\begin{matrix}{{\hat{S}}_{i} = {\frac{S_{i{({tooth})}}}{S_{i{({ref})}}}R_{{ref}{({\lambda\; i})}}}} & (3)\end{matrix}$wherein R_(ref(λi)) is the known reflectance of the reference target 28at the peak value of LEDi. The corrected measurement value Ŝ_(i) removesfirst-order sources of measurement variability and eliminates dependenceon the scale of intensity or device response. From the correctedmeasurement value Ŝ_(i), subsequent derivations can be used to obtainvisual color values to provide a highly accurate color mapping.

Each of the tristimulus color values X, Y, Z (hereafter represented byX_(q) (q=1, 2, 3), where X₁=X, X₂=Y, and X₃=Z) for each pixel can becalculated from the tooth reflectance in the following way:X _(q) =∫I(λ)R _(tooth)(λ) x _(q)(λ)dλ  (4)wherein the value of x _(q)(λ) is the corresponding visual colormatching function of the standard observer, and I(λ) is the spectraldistribution of the light source under which the tooth is viewed.

Value R_(tooth)(λ) is not directly measured by color measuring andspectral reflectance mapping apparatus 40, but can be estimated from theN corrected measured values Ŝ_(i), through the process of datainterpolation. For a simple linear interpolation,

$\begin{matrix}{{R_{tooth}(\lambda)} = {\sum\limits_{i}^{N}{{\alpha_{i}(\lambda)}{\hat{S}}_{i}}}} & (5)\end{matrix}$wherein α_(i) are the interpolation coefficients.

Equation (5) can be used to reconstruct the full spectral reflectancecurve at the pixel. The result of this processing can then be used tocalculate the tristimulus values according to equation (4).

Instead of carrying out equation (5) and equation (4) as two separatesteps, these two equations can be combined to directly relate thetristimulus values to the corrected measured signal Ŝi. In combiningequations (5) and (4), there is only a need to evaluate the twoexpressions at discrete wavelengths λ=λ_(k), corresponding to the peakwavelengths of the LEDs or other light sources, so that α_(i)(λ)=α_(ik).As a result, the tristimulus color values can be computed directly fromthe corrected measured values Ŝi, in the form:

$\begin{matrix}{{X_{q} = {\sum\limits_{k}\;{{I\left( \lambda_{k} \right)}{{\overset{\_}{x}}_{q}\left( \lambda_{k} \right)}\Delta\;\lambda{\sum\limits_{i = 1}^{N}\;{\alpha_{ik}{\hat{S}}_{i}}}}}},{or},} & (6) \\{{X_{q} = {\sum\limits_{i = 1}^{N}\;{E_{qi}{\hat{S}}_{i}}}},} & (7)\end{matrix}$wherein the matrix equivalent.

$\begin{matrix}{E_{qi} = {\sum\limits_{k}\;{\alpha_{ik}{I\left( \lambda_{k} \right)}{{\overset{\_}{x}}_{q}\left( \lambda_{k} \right)}{\Delta\lambda}}}} & (8)\end{matrix}$can be calculated from the particular interpolation kernel that is used,the known characteristics of the illumination for viewing the tooth, andthe visual color matching function.

Employing the matrix multiplication relationship of equation (7),corrected measured values obtained from color measuring and spectralreflectance mapping apparatus 40 can be used to calculate colortristimulus values for tooth under any lighting condition with knownspectral energy distribution. Applying this procedure to both teeth andshade-matching tab will yield two sets of tristimulus values from whichthe best match can be found. Alternatively, the tristimulus values canbe converted to a different visual color space, such as the CIELAB orHSV color space, for finding the closest match between tooth andshade-matching tab.

The above approach has been described for a monochrome broadband sensorarray. The same method can also be used if a conventional RGB sensorarray is used to provide a signal as each LED is energized. In the caseof an RGB color sensor, the color channel with the highest signal levelfor a particular LED illumination can be used, and the tristimulusvalues are still calculated according to equation (7). If all threecolor channels of the sensor are used for each LED illumination,tristimulus values can be calculated from the corrected measured valuesin a manner similar to equation (7), but with an additional matrixmultiplicand, which is determined by how the signals of the multiplecolor channels are combined.

As disclosed in this invention, the task of obtaining aspectrophotometric color mapping of the tooth surface is simplified, andthe cost of providing this data is significantly reduced overconventional alternatives.

LEDs are advantaged as light sources for obtaining spectral reflectancedata according to the present invention. In a preferred embodiment, theLED emits most of its light over a range of wavelengths that is lessthan about 40 nm. Other types of narrow band light sources couldalternately be used to provide the needed illumination for reflectancemeasurements. Alternate types of light source include filtered lightfrom a broadband light source, such as a lamp, for example. Four LEDsare shown in the example of FIG. 3. In general, at least 3 light sourcesof different wavelengths should be used; employing more than 4 lightsources provides the advantage of increase number of measured points forgenerating the spectral reflectance curve.

Because the apparatus and methods of the present invention provide amapping of spectral reflectance values, they give more accurateinformation about the true color of the tooth than is available whenusing conventional colorimetric or RGB-based measurement methods.Because spectral reflectance contains complete color informationindependent of illuminant, the color data that is obtained is notsubject to error due to metamerism. By using LEDs or other smallsources, without the need for gratings or other devices, the apparatusof the present invention can be packaged in a compact fashion, at lowcost. For example, color measuring and spectral reflectance mappingapparatus 40 can be packaged as an intraoral camera.

Reference target 28 can be any suitable type of reference patch forproviding a reference image for multi-color imaging. A white or graypatch can be used, as well as some other patch having uniform spectralcontent. Target 28 can be separately imaged, or it can be positionedalongside the tooth, at approximately same distance from and within theimaging field of view of sensor array 44, such as in a fixed position,or may be pivoted into position, such as by energizing an actuator, forreference imaging as needed in the imaging cycle.

A number of computation functions are required, such as forinterpolation, for obtaining a matrix equivalent, for obtainingtristimulus values, and for storing and displaying results, for example.It can be appreciated that these functions can be provided by a controllogic processor 38 provided with or configured to interact with thecolor matching and mapping apparatus of the present invention. Storedinstructions, for example, configure the processor logic circuitry toexecute the color mapping data access, calculations, and outputfunctions described above. Any of a number of types of control logicprocessor devices, such as a dedicated computer workstation, personalcomputer, or an embedded system that employs a dedicated data processingcomponent (such as a digital signal processing component) could be usedfor computation functions. Control logic processing apparatus access anelectronic memory for data storage and retrieval. An optional displaydevice can also be provided for displaying color match and color mappingresults.

As compared to other color-matching solutions, the method and apparatusof the present invention provide a spectrophotometric or spectralreflectance color mapping that effectively stores, for each pixel in thetooth image, actual spectral reflectance data values R_((tooth)) thatcan be used to accurately profile the color of the tooth surface. Bycomparison against conventional methods that rely on RGB conversiontransform or colorimetric measurements, the methods of the presentinvention are capable of generating a mapping that includes aconsiderable amount of data for each pixel of the tooth image. This isrepresented schematically in FIGS. 7A and 7B. FIG. 7A shows the colordata gathered for a single pixel P using conventional color-measuringapproaches. In the example shown, a single data value is provided foreach of the Red, Green, and Blue color planes. By contrast, FIG. 7Bshows the nature of the spectral reflectance mapping data that isobtained for each pixel using the methods of the present invention.Here, each pixel P effectively has an associated spectral reflectancecurve that provides a substantial amount of information on its actualcolor content, from which tristimulus values X, Y, Z or other color datacan be derived. In practice, only a small amount of data may actually bestored in memory for each pixel P following this processing; thespectral reflectance curve itself can be reconstructed using the storedmatrix of coefficients obtained as described earlier. In memory storage,then, each pixel can be associated with a set of spectral reflectancevalues and, optionally, also have a link or other identifier for aninterpolating matrix that is capable of re-creating the full spectralreflectance curve for that pixel. Alternately, calculated tristimulusvalues, CIELAB values, or values in other standard color space could bestored for each pixel P.

One advantageous result of having spectral reflectance data values isimproved color matching between dental objects, such as between a toothand a prosthetic device or material. Using the data acquisition andprocessing sequence of the present invention, color matching can beaccomplished using any of a number of mathematical techniques. In oneembodiment, the spectral reflectance curve for a dental object A iscompared with the spectral reflectance curve for a dental object B anddifferences between the two curves are evaluated for closeness of fit orother metric. For example, the overlap area between two curves overdifferent wavelength ranges can be evaluated. In other embodiments,tristimulus data values or CIELAB color values are obtained for each ofthe dental objects and are compared.

The method and apparatus of the present invention employ a sensor array34 as the detector in order to obtain a mapping with highly accuratecolor information. This component can include an arrangement of multipleCMOS or CCD sensors, typically assigned one per pixel. In oneembodiment, a broadband monochrome sensor array is used. However, it isalso possible to employ a sensor array that is configured for R, G, Bcolor sensing, or configured for some other color space characteristics.Methods of obtaining spectral reflectance data in the present inventioncan be similarly applied to such devices, as has been discussed earlier.It should be noted that pixel spacing can be varied for color matching,so that multiple sensor sites in sensor array 44 are grouped orclustered together, such as to obtain an averaged value, for example.Because the color measurement apparatus of the present invention uses animaging sensor array, the same device that is used for conventionalintra-oral color imaging can be configurable for both imaging and colormeasurement modes of operation. Referring to the schematic block diagramof FIG. 3, for example, color imaging can be performed by changing thepattern of illumination from the illumination apparatus 24, theresolution of the sensor array 34, and the color processing providedfrom control logic processor 38. A mode switch (not shown) or modecontrol command issued from an operator interface can be provided inorder to set the operating mode of apparatus 40 for either conventionalimaging or tooth color mapping.

Illumination apparatus 24 of the present invention employs multi-colorLEDs in one embodiment. However, other light sources that can providemultiple color illumination could alternately be employed, includingother types of solid-state light sources or more conventional lamps orlamps equipped with color filters.

Initial and periodic calibration of color measuring and spectralreflectance mapping apparatus 40 is needed in order to compensate forcomponent aging and drift, so that the profile of each LED inillumination source 12 can be maintained and regularly updated.

The invention has been described in detail with particular reference toa presently preferred embodiment, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the invention. The presently disclosed embodiments are thereforeconsidered in all respects to be illustrative and not restrictive. Thescope of the invention is indicated by the appended claims, and allchanges that come within the meaning and range of equivalents thereofare intended to be embraced therein.

PARTS LIST

-   10. Imaging apparatus-   12. Illumination source-   14 b, 14 g, 14 r, 14 y. LED-   18. Capture apparatus-   20. Tooth-   22. Color reconstruction apparatus-   24. Illumination apparatus-   28. Reference target-   30. Imaging apparatus-   32. Lens-   34. Sensor array-   36. Color calibration transform-   38. Control logic processor-   40. Color measuring and spectral reflectance mapping apparatus-   44. Sensor array-   46. Spectral reflectance interpolation model-   S110. Illumination step-   S120. Image capture step-   S130. Obtain values step-   S140. Interpolation step-   S150. Calculation step-   λ1, λ2, λ3, λ4. Wavelength band-   P. Pixel

1. A method for obtaining a color mapping for a dental object, themethod executed at least in part by a control logic processor,comprising: directing illumination toward the dental object over atleast a first, a second, and a third wavelength band, one wavelengthband at a time; capturing, on an imaging array, an image of the dentalobject at each illumination wavelength band to form a set of images ofthe dental object; for each of a plurality of pixels in the captured setof images: (i) obtaining an image data value for the pixel correspondingto each of the at least first, second, and third wavelength bands; and(ii) calculating a plurality of interpolated image data valuesproportional to the spectral reflectance of the dental object, accordingto the obtained image data values and according to image data valuesobtained from a reference object at the at least first, second, andthird wavelength bands; obtaining spectral distribution data for aviewing illuminant; reconstructing the visual color of the dental objectaccording to the calculated interpolated image data values and theobtained spectral distribution of the viewing illuminant; and storing atleast the reconstructed visual color as data in a computer-accessiblememory.
 2. The method of claim 1 wherein directing illuminationcomprises energizing one or more LEDs.
 3. The method of claim 1 whereinthe imaging array is a CMOS or CCD sensor array.
 4. The method of claim1 wherein the imaging array is a broadband or monochrome sensor array.5. The method of claim 1 wherein calculating the plurality ofinterpolated image data values comprises determining a set ofinterpolation coefficients for the obtained image data values.
 6. Themethod of claim 5 wherein calculating the plurality of interpolatedimage data values further comprises generating a reflectance curve as afunction of wavelength from the set of interpolation coefficients. 7.The method of claim 5 further comprising calculating visual color valuesfrom the set of interpolation coefficients.
 8. The method of claim 7wherein calculating visual color values comprises computing a product ofthe set of interpolation coefficients and the obtained image datavalues.
 9. The method of claim 6 further comprising calculating visualcolor values from the reflectance curve.
 10. The method of claim 1further comprising capturing, on the imaging array, an image of thereference at each illumination wavelength band.
 11. The method of claim10 wherein the reference object is captured within the same image asthat of the dental object.
 12. The method of claim 10 wherein thereference object is captured as a separate image from that of the dentalobject.
 13. The method of claim 10 wherein the reference object is atest patch associated with an intra-oral imaging device.
 14. The methodof claim 10 wherein the reference object is a test patch that is astandalone target.
 15. The method of claim 10 wherein the image of thereference object is used to correct the set of image data values of thedental object.
 16. The method of claim 1 wherein the imaging array is acolor sensor array.
 17. The method of claim 1 wherein the dental objectis taken from the group consisting of a tooth, a color matching guide, adental prosthesis, and a dental material.
 18. The method of claim 1wherein the visual color values are tristimulus values, CIELAB values,HSV values, or other equivalent values in a standard color space. 19.The method of claim 6 wherein the dental object is a first dental objectand further comprising obtaining the reflectance curve for a seconddental object and comparing the reflectance curve for the first andsecond dental objects in order to calculate a color match.
 20. Themethod of claim 1 wherein the dental object is a first dental object andfurther comprising comparing reconstructed visual color with a seconddental object to calculate a color match.
 21. An apparatus for obtaininga color mapping for a dental object, comprising: an illuminationapparatus energizable to direct illumination of at least a first, asecond, and a third wavelength band, one wavelength band at a time,toward the dental object; a monochrome image sensor array having broadspectral response over at least the first, second, and third wavelengthbands, that is disposed with respect to an optical system and that isactuable to capture an image of the dental object at each illuminationwavelength band to form a set of images of the dental object; areference target that is disposed in the object field of the opticalsystem and substantially in focus with respect to the image sensorarray; and a control logic processor operatively connected with theillumination apparatus and monochrome image sensor array and responsiveto stored instructions to energize the illumination apparatus tosequentially provide the at least first, second, and third wavelengthbands and further responsive to stored instructions to capture and storean image at each wavelength band, to calculate image data values foreach pixel, to perform data interpolation to generate interpolated imagedata values used for spectral reflectance mapping, and to perform visualcolor calculations according to a predetermined illuminant, wherein anelectronic memory is in communication with the control logic processorfor storing at least the spectral reflectance values for each pixel. 22.The apparatus of claim 21 in the form of an intraoral camera.